The Efeect of Contention on the Scalability of Page-Based Software Shared Memory Systems

We demonstrate the profound effects of contention on the performance of page-based software distributed shared memory systems, as such systems are scaled to a larger number of nodes. We argue that applications that suffer from increases in memory latency due to contention and imbalances in protocol load scale poorly Furthermore, we show that there is a relationship between protocol imbalance, contention, and memory latency, whereas higher imbalances tend to aggravate contention, which in turn increases memory latency. Moreover, we show that applications that exhibit good scalability have constant memory latency. We experimented on two widely used page-based protocol, Princeton’s home-based LRC and TreadMarks. For most of our programs, these protocols were equally affected by latency increases and achieved similar performance. Where they differ significantly, HLRC’s avility to manually eliminate load imbalance was the largest factor accounting for the difference. Finally, we introduce Adaptive Striping, and improvement to the Tmk protocol that eliminates load imbalance automatically.